ci(flow): overlap guard gates regressions instead of failing outright - #302
Merged
Conversation
…outright The guard has been red on master since the SGCR migration — 24/31 boards clean — so it gated nothing. #298, #299 and #300 all merged through it, and #300 shipped titled "edge-over-node eliminated" while seven boards still had edge-over-node. A check that is always red is indistinguishable from a check that is always green, except that it also hides the next real regression. Record the known-bad boards in e2e/overlap-baseline.json and fail only on boards outside it. Everything not listed must still render with zero overlaps, so a clean board breaking is a hard failure again. Deliberately board-level rather than violation-level: the exact node ids differ between macOS and Linux CI (text metrics move the layout — ex_swimlane fails on CI and passes locally), so pinning ids would fail constantly off-CI. To keep that coarseness from hiding drift, violations are now printed in FULL rather than truncated to three, so a known-bad board getting worse is visible in review even though it doesn't flip the exit code. Flakiness is never baselined: an entry records a bad-but-stable layout, and a board whose geometry moves between runs still fails. The guard also reports when a listed board renders clean, so the ledger only ever shrinks.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ci(flow): make the overlap guard gate regressions instead of failing outright
The guard has been red on master since the SGCR migration — 24/31 boards clean —
so it gated nothing. #298, #299 and #300 all merged through it, and #300 shipped
titled "edge-over-node eliminated" while seven boards still had edge-over-node.
A check that is always red is indistinguishable from a check that is always
green, except that it also hides the next real regression.
Record the known-bad boards in e2e/overlap-baseline.json and fail only on boards
outside it. Everything not listed must still render with zero overlaps, so a
clean board breaking is a hard failure again.
Deliberately board-level rather than violation-level: the exact node ids differ
between macOS and Linux CI (text metrics move the layout — ex_swimlane fails on
CI and passes locally), so pinning ids would fail constantly off-CI. To keep
that coarseness from hiding drift, violations are now printed in FULL rather
than truncated to three, so a known-bad board getting worse is visible in review
even though it doesn't flip the exit code.
Flakiness is never baselined: an entry records a bad-but-stable layout, and a
board whose geometry moves between runs still fails. The guard also reports when
a listed board renders clean, so the ledger only ever shrinks.